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Abstract. Consider the set of vectors over a field liaving non-zero coefficients only in a 
fixed sparse set and multiplication defined by convolution, or the set of integers having non- 
zero digits (in some base b) in a fixed sparse set. We show the existence of an optimal (resp. 
almost-optimal in the latter case) 'magic' multiplier constant that provides a perfect hash 
function which transfers the information from the given sparse coefficients into consecutive 
digits. Studying the convolution case we also obtain a result of non-degeneracy for Schur 
functions as polynomials in the elementary symmetric functions in positive characteristic. 



1. Introduction AND Motivation 

Suppose n > 0, and let D = {do,di,. . . ,t/„-i} be a set of indices such that = do < 
di < ■ ■ ■ < ii„_i. For a field F and = d„_i + 1, let be the standard vector space with 
basis ej for < j <N equipped with convolution multiplication 

i+k=i 

0<j,k<N 

and let F [D] be the subspace spanned by e^/g , . . . , e^;^ , , which is formed by the vectors with 
non-zero coefficients only in the indices di for < / < n — 1 . Similarly given a positive 
integer b let [D] be the set of integers that can be written as Y!i=Q ^ib'^' for some a; G 
{0, 1 , . . . , — 1 }, i.e. the set of numbers such that their base-/? representation only contains 
non-zero digits in positions that belong to the set D. We study the existence of constants 
that can be used as multipliers to transfer the information stored in the sparse digits of an 
element of F [D] or of [D] into a smallest possible set of consecutive digits, providing a 
perfect hash function. 

Motivation for this kind of questions is provided by a technique used by many state-of- 
the-art chess playing programs (|2l[8l, based on the concept of 'bitboards', that are numbers 
whose base-2 representation is interpreted as an occupancy information of some kind, or 
more generally to store a — 1 information for each square, having previously established 
a correspondence between a range of digits and the squares on the board. The technique 
in question, known under the name of 'magic bitboards', is a quick way to generate all 
possible attacks for sliding pieces, such as rooks and bishops. The bitboard containing 
occupancy information for all pieces is transformed with a bitwise-and to a bitboard whose 
only digits that may be different from zero are those corresponding to possible obstructions 
on the path of the sliding piece. This information about the obstructions, which is stored 
in a small set of sparse digits, is then mapped via a multiplication by a 'magic number' 
to a set of consecutive digits, which is then used as index in a lookup table to recover a 
pre-calculated information about the possible attacks. 

While in the case of chess programs a database of very efficient multipliers has already 
been computed and is publicly available, we investigate the existence of multipliers that 
provide perfect hashing functions in a more general setting. In the convolution case we 
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provide an optimal result, which shows that it is possible to transfer the information stored 
in any number of sparse digits into the same number of consecutive digits, and which in- 
cidentally provides a result about values of Schur functions as polynomials in elementary 
symmetric functions. On the other hand, in the case of base-fe integers it is not always pos- 
sible to have a multiplier providing a hash into the same number of digits (D {0, 1,2,4,6} 
providing the smallest counterexample for b = 2, as can be checked with a simple com- 
puter program), and we provide a linear estimate of the number of consecutive digits that 
are required to ensure the existence of such a map. 

While this kind of hashing cannot be directly compared to universal hashing (see fgllTOl 
13, and |, 11 1 in particular) because of its more restricted scope, it is still possible to compare 
the results about its effectiveness, and this is done below. See also Q. 

Acknowledgements. We wish to thank Vincenzo Mantova for the time enjoyed discussing 
this and related questions. We also thank the reviewers for suggesting relevant references. 



2. The convolution case 

In this section we consider the convolution case. The operation of taking the convolu- 
tion multiplication with a fixed vector {oq, a^- 1 ) can be expressed by a lower triangular 
Toeplitz matrix A = {ai-j)Q<ij<N, where we have put a,- = for negative / (the full con- 
volution with a vector (a_Af+i , • • • i<^N-i) is expressed by a general Toeplitz matrix, but as 
shown below we can restrict to the class of lower triangular matrices). In the same way, 
when restricting the output to a set of coefficients we shall consider the matrix formed by 
the corresponding selection of rows of A. We will show that for a good choice of the a, the 
matrix formed by the last n rows of A defines a one-to-one function from F[D] to F". 

The operation of taking the convolution with a fixed vector, or multiplying by a Toeplitz 
matrix, is also equivalent to the multiplication by a polynomial in a ring of polynomials, 
and its properties as hash function are well known, as well as fast algorithms, see EKil. 
Note also that taking the convolution with a vector with entries in the set {0, 1} is actually 
obtained by addition of selected entries. 

Theorem 1. For every set D of cardinality n, there exists a lower triangular Toeplitz matrix 
such that its last n rows define a one-to-one fi^nction from F[D] to F", and furthermore its 
entries can be taken in the set {0, 1 }. 

Proof Let 5 = N ~n, and for a vector (aq, ... ,aN^i) that is going to be detemined let A 
be the associated rectangular Toeplitz matrix (ag+i- ,) o<i<« corresponding to the selection 

0<j<N 

of the last n coefficients after taking the convolution. Let B be the nx n matrix obtained 
from A selecting the columns with indices do,... ,dn-i, which defines the linear map on 
the basis of F[D] formed by the ej. 



B — {as+i-d,,)o<i<n 
0<k<n 



\^5+n-l-do ^S+n-l-di ■■■ ^S+n-l-d„_J 



We will now consider a sequence of kx k minors, for k — 1 , . . . , n, where each minor will 
contain the previous one, and inductively change some of the a,- ensuring at the ^-th step 
that the determinant of the ^-th minor is non-zero, while leaving unchanged the coefficients 
of the minors considered in the previous steps. 

Let the ^-minor Bj^ be obtained taking the first k rows, and a range of columns rji,ri^ + 
\,. . . ,rii-\-k — 1, where Q < ri^ <n — k is, the biggest integer such that each column of 
will contain at least one a,- with / > 0. It is an easy consequence of B being a selection of 
columns from the Toeplitz matrix A that the set of integers r^. satisfying the above condition 
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is always non-empty, and that the set of columns selected for Z?<. is the same set that was 
selected for B/t^i with one column added either on the left or on the right. 

Let's begin the induction putting ao = 1 and a; = for all / ^ 0. For k = 1, change 
also as-ii^^ to be equal to 1. Let now k he > 1. When the columns of Bi^ are those of 

Bj^^i plus one column on the right, we have that the matrix B/. = [ ^'^^^ ^ ) is block 

lower triangular, with one block equal to B^^ i , and the other block being formed by the 
element ao = 1, and B/^ is non-singular On the other hand, when one column is added on 

the left, Bii is of the form ^^-i \ ^ ^jjgj-g £= § ^ k + 1 — dr. is the biggest index 

\ae * J 

appearing in Z?^. (in fact, the indices are decreasing while moving right along a row or up 
along a column). Consequently, considering the Laplace expansion of the determinant of 
Bf; along the first column 

k-l 

dstBk = ae-k+i ■ Ci,i{k) + ae ■ detB<._i (1) 

1=1 

where for each Cij{k) is the i,j cofactor of the matrix B^, we can select an appropriate 
value for a( which makes the determinant non-zero, while changing only the bottom left 
entry of B^. Repeating this step up to A: = n we have the theorem. □ 

Remark 1. Since no division is involved the above proof works in any ring with 1, but it 
does not ensure the resulting matrix B to be invertible, only to have non-zero determinant. 
Alternatively, it is possible to allow general a,-, and solve inductively the ([T]l in ag to ensure 
that detBk ^ I at each step, obtaining that B is invertible because the determinant is an 
invertible element of the ring. 

IfF is a local ring (i.e. a ring with only one maximal ideal) with maximal ideal M we 
can still take the a, in {0, 1}, and make all the determinants of the B^ invertible: consider 
the ([T]l and call S the sum, since in a local ring the invertible elements are precisely those 
not in M and we assume detBj-_i to be invertible, we cannot have both S ^ M and S + 
detBk-i £ M or we would also have detB^-i G M, and hence putting a^ equal to either 
or 1 we obtain that AttB^ ^ M, i.e. that it is invertible. 

This has a practical consequence: take for instance F to be the set of integers modulo 
2*^, it is a local ring and hence we have the existence of a magic multiplier with entries in 
the set {0, 1 }, and taking the convolution is actually the addition of selected enties. 

It is possible to observe that the Schur functions that are well known in Algebraic Com- 
binatorics (see f3','6l) for their combinatorial properties and connections with the charac- 
ters of the symmetric group can be expressed as a determinant of a special matrix hav- 
ing the elementary symmetric functions as coefficients via the Jacobi-Trudi identity (also 
known as "determinant formula"), which has the same form as the transpose of the matrix 
B considered above. In particular, if sx is the Schur function associated to the partition 
X = (Ai , A2, . . . ) and X' is the conjugate partition, and e, is the /-th elementary symmetric 
function for / > 0, we have the formula 

Sx ^det{exi^i+j)i<ij<„ 

expressing the Schur function sx as a uniquely determined polynomial in the e, having 
integral coefficients. Since the set D considered in Theorem[T]is arbitrary, for each partition 
of length n we chose D ensuring that 5 — dk- 1 = Xj^ — k for 1 < k < n, making the matrix 
B equal to the transposed of above matrix evaluated with e,- — a, for all /. It follows that 
the polynomials expressing the sx in terms of the e, assume a non-zero value when the e; 
are replaced with opportune values in the field F (note that this is not true for a general 
polynomial over a field, as the example x^ —x over ¥p shows). Hence we have 
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Theorem 2. For a partition A, consider the Schur function as a polynomial in the 
elementary symmetric functions e„ considered as indeterminates. Then it takes a non-zero 
value after substitution of the variables e,- with appropriate elements of the field F, which 
moreover can be taken in the set {0, 1}. 

3. The arithmetic case 

The case of base-/? digits of integers seems to be much more difficult, and we give a 
linear bound on the number of digits required to ensure the existence of an opportune mul- 
tiplier. Let Z(^,) be the set of rational numbers that can be written as r/b'^ for some integers 
r,k, or equivalently that have a finite base-/? expansion. For such an a = Y.i<M'^i^' G ^(fc)' 
define [«](:,„, as the m-tuple (a/i+m_i , . . . , Q/t+i , G {0, 1 ,...,/>— 1 j™. We can now state 

Theorem 3. Let D = {do, . . . ,dn-i} be a set of indices ~ do < ■ ■ ■ < d„-i having cardi- 
nality n. Then for 

m=\log,{{2b-ir-l)-] 
there exist a multiplier jU € Z and a G N such that the map from Z;, [D] fo {0, 1 , . . . , /> — 1 }'" 
defined by a |fl • ^\k^m injective. Furthermore, ifD contains some consecutive integers 
and is formed by the union of the integral intervals {c,-,c, + 1, . . . ,c,+^, — l}/or ; = \ ,. . . ,k 
and £i>l, we can take 

log, (jl{2b'^-l)-l 

It is possible to compare this estimate with what can be obtained using universal hash- 
ing: when a hashing function is randomly chosen in a universal class (i.e. h{x) = h{y) 
with probability at most 1 /b"\ what can be done when the output is formed by at least 
m digits, see IfTTI ) we have that the probability of h being one-to-one on Zfe[D] is at least 
1 - Qb-"\ and we deduce the existence of a good hash function when m is > 2n. The 
above result is sharper because it just requires m to be about n log, (2/?— 1), which is always 
smaller than 2n and its ratio with n approaches 1 as b grows. 

Proof. We will prove the second estimate, as the first one can be obtained taking n intervals 
of length £i = I. Each element in Z, [D] can be written as 

k 
i=l 

with < A,- < b^' — 1, for each 1 < / < A:. Consequently the difference of two elements 
a, a' G Z/,[D] can be written as 

k 

a~a' = Y,iAi~A'i)b'-, 

1=1 

where —b^' + 1 < A, — Aj < b^' — 1, for each 1 < / < A;. In particular the number A of 
positive differences of two elements of Z, [D] is at most 

nli{2b'--i)-i 

2 

For any real number r, let T,. = M/rZ, and let TZr '-M.^ T,- be the projection map. For 
z G Z \ {0} the map j3 : given by multiplication by z is measure-preserving, i.e. 

for each measurable X CTr the measure of j3^' (X) is equal to the measure of X. 

Let now b'" be a power of b which is > 2A. The measure in T,m of the set = 
1])) of the 'bad' A G Tb>. such that zA G ;/r/y-([-l, 1]) is equal to 2/b"', 
supposing the measure of T,™ to be normalized to 1 . Since the number of positive and 
non-zero differences is < b"'/2, and clearly U-^ = 11^, we have that the union of all the 
Ua-a' for all distinct a,a' ^ '^p[D] cannot be all T,™. Consequently since Z(,) is dense in 
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M there exist an element in v e Z(^) which falls out of all the Ua-a' when reduced modulo 
b^, being the above union a closed set. We have that av and a'v differ by at least 1 after 
reduction modulo b"', and hence the map a^\a- v|o,m is injective. 

Multiplying by the smallest power of divisible by the denominator of v obtain an 
integer = vfo* with the required properties with respect to the map a>-^la- ^l}k,m■ ^ 
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